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Summary of Review 

A Friedman Foundation report attempts to find empirical support for the con- 
tention that competition from private schools, through voucher programs, im- 
proves the effectiveness of public schools. In the first year of Ohio’s new Ed- 
Choice voucher program, the report claims to have found substantial academic 
gains at public schools exposed to the possibility of losing students to vouchers. 
Despite being presented as scientifically rigorous, the report suffers from serious 
methodological shortcomings. The analysis uses weak variables and an incorrect 
approach to measuring academic gains and tries to make claims based on cherry- 
picking uneven results. Moreover, even accepting the study’s analysis, it produces 
a finding very much at odds with the author’s intent: that vouchers are not likely 
to close the achievement gap between high- and low-performing schools. 
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Review 



I. Introduction 

Much of the research on programs that 
provide public funding to assist stu- 
dents in moving from public schools to 
private schools has focused on the im- 
pact on the student using the voucher, 
and arguments have raged about 
whether these programs actually boost 
achievement for those students transfer- 
ring to private schools.^ In comparison, 
the question of the effects of vouchers 
on public schools has less often been 
the subject of rigorous empirical analy- 
sis. These effects, however, should be a 
primary concern because most Ameri- 
can students remain in public schools, 
even in areas with established voucher 
programs. If voucher policies are to 
have wider impact beyond the small set 
of students who use them (and for 
whom the benefits are not clear), then 
their effects must extend to public 
schools. 

Moreover, the positive effects that pol- 
icy makers would seek must overcome 
any negative effects of spiraling de- 
clines in public school enrollment and 
funding potentially caused by vouchers. 
The overall effect should be the crea- 
tion of demonstrably better outcomes 
for the students remaining in public 
schools. This is the claim made by the 
most recent report of the pro-voucher 
Friedman Foundation for Educational 
Choice, a report that serves as a timely 
reminder that these programs are also 
advanced with the stated purpose of 
generating the competitive incentives 
necessary to spur public schools to be- 
come more effective. The report is 
timely because it comes in the wake of 
a controversy ignited when other 



prominent voucher advocates publicly 
repudiated the movement on exactly the 
grounds that the report tries to reclaim 
here: the apparent failure of vouchers to 
generate improvement in public 
schools.^ In that respect, the report is 
particularly interesting because it at- 
tempts to find empirical support for a 
key plank of the voucher platform just 
as other advocates are citing a lack of 
such evidence. 

In theory, vouchers for private schools 
not only provide benefits for the more 
active, discerning, or knowledgeable 
consumers that use them, but also — as 
“a rising tide lifts all boats” — create 
more competitive conditions that lead 
to higher quality educational options for 
all students when a program is in 
place. ^ As Milton Friedman noted, 
competitive market incentives often 
lead to quality improvements for an 
elite few, but these improvements tend 
to filter down to the “basic product” as 
well if they are adopted by mass pro- 
ducers, thereby benefiting the many.^ 
Faced with the loss, or the threat of the 
loss, of students and the funding at- 
tached to those students, public schools 
may embrace more effective practices 
in order to attract and keep students 
who could otherwise use a voucher to 
transfer to a private school. 

While the immediate effect on voucher 
students has been much studied, there 
has also been some research done on 
this secondary effect on public schools. 
Much of this research has been con- 
ducted by pro-voucher organizations 
such as the Friedman Foundation. (That 
research is discussed in greater detail 
later in this review.) 
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Given the generally positive findings of 
pro- voucher researchers, it was notable 
when, over the past year, major cham- 
pions of vouchers highlighted the fail- 
ure of vouchers to spark improvement 
in public schools. Writing in a publica- 
tion of the pro-voucher Manhattan In- 
stitute, Sol Stern decided that school 
choice is not improving schools, la- 
menting that “the evidence is pretty 
meager that competition from vouchers 
is making public schools better.”^ Ear- 
lier, the pro-voucher Wisconsin Policy 
Research Institute released a report 
finding little academic improvement in 
Milwaukee’s public schools since that 
city’s voucher program began almost 
two decades ago.^ One of the prime 
movers of the voucher program in Mil- 
waukee, Howard Fuller, indicated, “we 
may have oversold that point... I think 
that any honest assessment would have 
to say that there hasn’t been the deep, 
wholesale improvement in MPS that we 
would have thought.”^ The Friedman 
Foundation’s report appears at a time 
when there is a growing hole in the 
dike. 

II. The Report’s Findings and 
Conclusions 

This new report from the Friedman 

o 

Foundation examines Ohio’s new Ed- 
Choice program, a voucher plan that 
provides $4,375 ($5,150 for high 
schoolers) for up to 14,000 students in 
“chronically under-performing” public 
schools to attend a private school par- 
ticipating in the program.^ The state- 
wide program began in 2005, three 
years after the US Supreme Court ruled 
5-4 that Ohio’s voucher program in 
Cleveland was constitutional. 

Greg Forster, a fellow at the Friedman 



Foundation and author of the report, 
examined student growth in “voucher- 
eligible public schools”'* from 2005- 
2006 (as students were applying for the 
program) to 2006-2007 (the first year 
the program was in operation) for 
grades 3-8, finding “substantial aca- 
demic improvements” in those schools 
threatened by vouchers, which he as- 
cribes to the competitive effects gener- 
ated by the voucher program (p. 5). The 
report, which was released jointly by 
the Friedman Foundation and nine other 
pro-voucher organizations, notes bene- 
ficial effects in some grades and “no 
negative effects” in the others (p. 5). It 
posits that, because the program is 
likely to grow and since gains may ac- 
cumulate over time, EdChoice may go a 
long way in closing the achievement 
gap between the lowest and highest per- 
forming schools. Thus, the report con- 
cludes that it provides further evidence 
that vouchers do not harm public 
schools, but in fact “vouchers improve 
academic outcomes at public schools. 
Vouchers allow fa mi lies to choose the 
right schools to meet their children’s 
needs and introduce competitive incen- 
tives for improvement that are lacking 
in the traditional government-run edu- 
cation system” (p. 5). 

III. Rationales Supporting 
Findings and Conclusions 
OE THE Report 

The report uses data from the state of 
Ohio on achievement and basic demo- 
graphics to compare academic gains in 
“voucher-eligible” schools to all public 
schools, controlling for several factors. 
The analysis wisely seeks to factor in 
the presence of charter (or, in Ohio, 
“community”) schools, which otherwise 
could confound efforts to deter mi ne if 
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any competitive effect was actually due 
charter schools and not vouchers. The 
report also appropriately focuses on 
grades 3-8, since achievement data for 
secondary grade-levels were not avail- 
able, and it excludes from the analysis 
schools with smaller grade cohorts so 
that individual students could not be 
identified. The report then uses linear 
regression to assess the impact of 
voucher eligibility on academic gains 
since the program began, comparing 
voucher-eligible schools to all Ohio 
public schools at grades 3-8. To con- 
sider the possibility of regression to the 
mean — schools with unusually low 
scores at Year 1 might be more likely 
than mid-scale schools to show 
growth — the report includes a second 
analysis, focusing only on voucher- 
eligible schools in state-determined 
“very poor urban districts” compared to 
other public schools in such districts. 

While I review the data and methods 
below, it is important to note that the 
report advances from several assump- 
tions, not all of which are supported or 
even considered in the analysis. The 
report assumes that any gains in 
achievement result from schools’ or- 
ganizational effects, ignoring alterna- 
tive explanations — other influences on 
student achievement are well- 
documented in the research literature, 
including peer effects and student and 
school socioeconomic status (SES).'^ 
The report’s analysis implements basic 
demographic controls that are inade- 
quate for addressing SES, as described 
below, and since it relies on school- 
level, rather than individual student- 
level, data it is impossible to determine, 
as the report claims to do, if its analytic 
approach in fact “removes most of the 
impact of confounding variables such 



as demographic factors and unobserved 
characteristics” (p. 12). 

The report contends that vouchers allow 
parents “to hold schools accountable for 
teaching their students,” with the only 
other viable option for parents and stu- 
dents being “to move” to another dis- 
trict or school (p. 10). Leaving aside the 
fact that this claim ignores local democ- 
ratic mechanisms for holding schools 
accountable, the assertion is premised 
on a number of questionable assump- 
tions for voucher-based accountability 
to be workable: parents must have the 
information and knowledge about aca- 
demic effectiveness in different 
schools; they must have the time and 
motivation as well as resources (such as 
transportation) and ability to absorb 
search costs in order to act on their 
preferences; competitive incentives 
must be clear to schools, which must 
then be able to respond in ways that 
have a discernable impact on school 
effectiveness. Yet the evidence for 
many of these premises is less than 
clear. Still, the report ignores all of 
these pre-conditions, falling into the 
problematic “black box” approach of 
examining only outputs and assuming 
that vouchers caused any improvements 
in school effectiveness. 

The report’s conclusions are also based 
on the assumption that any voucher ef- 
fects on academic achievement are cu- 
mulative (see below). However, this 
ignores academic slips such as summer 
reading loss that detract from a simple 
accumulation of effects and are more 
likely to affect poorer children, such as 
the ones in these “very poor urban dis- 
tricts.” The report also assumes that 
the claimed impact of the program will 
increase, rather than dissipate, as the 
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program is expanded — as Ohio policy- 
makers seem likely to do. A contrasting 
possibility would be that the most obvi- 
ous and feasible responses to voucher 
competition — to the threat of losing 
students to EdChoice — were the first 
ones adopted by public schools, and 
later responses will have a diminishing 
rate of return. 

Furthermore, the report appears to em- 
brace a very simplistic model of market 
behavior. The report argues that 
“Where parents are empowered with 
school choice, schools that don’t ade- 
quately teach their students will lose 
them” (p. 10). Presumably, then, 

schools that do a better job of teaching 
will gain students. Yet the report pro- 
vides no evidence that this is happen- 
ing, and research does not necessarily 
bear out this assumption. For example, 
very few fa mi lies have used the exit 
option in the federal NCFB act to trans- 
fer their children from public schools 
designated as “failing,” and at the same 
time the fastest-growing schools — 
conservative Christian schools — are 
among the lowest performing types of 
schools.''^ This suggests that parental 
choices are influenced by a much 
broader set of factors than perceptions 
of “adequate” teaching. 

Instead of being empirically based, the 
report’s assumptions appear to be more 
statements of belief based in a rudimen- 
tary and simplistic view of economic 
behavior in markets for education. In 
fact, rather than applying a generic 
model of markets to education, it would 
have been more useful to think about 
the many different types of consumer 
markets, and how education does or 
does not exhibit attributes associated 
with those various markets, such as 



asymmetries of information, ease of 
entry, and repeat purchases. 

Finally, it has become fashionable for 
voucher advocates to claim that vouch- 
ers “do no harm” to public schools and 
do not “skim” better students from pub- 
lic schools. This largely unsupported 
contention (particularly considering the 
optimistic promises of a “panacea” 
made by earlier voucher advocates is 
repeated in this report. Since the results 
in the report are presented as averages 
for large sets of schools, we cannot tell 
from these data whether or not voucher 
dynamics harmed individual schools or 
students. In fact, there is growing evi- 
dence that choice programs such as 
charter schools — which are much more 
widespread than voucher programs — do 
in fact lead to increased sorting of stu- 
dents by race, SFS, and academic abil- 
ity. The study cited to refute this re- 
search — what the report calls the “best 
analysis” (p. 11) — was funded and con- 
ducted by pro-voucher organizations, 
and was not published in a peer- 
reviewed journal. It found that 
voucher students were demographically 
and academically similar to other stu- 
dents eligible for vouchers. In fact, by 
definition, there are qualitative and aca- 
demic differences between students ap- 
plying for vouchers to leave public 
schools and the students who do not 
apply, even if they are eligible. Fa mi lies 
applying for vouchers have demon- 
strated an otherwise unobserved com- 
mitment to investing time in their chil- 
dren’s education, and they differ on ob- 
servable factors as well.^^ 

IV. The Report’s Use 

OE Research Literature 

The report’s use of research literature is 
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somewhat misleading, highly selective 
and, in its use other advocacy literature, 
unabashedly incestuous. Specifically, 
the report overstates the empirical find- 
ings from earlier research. It uses a very 
selective and one-sided review of pre- 
vious studies to support its assertions, 
ignoring the more respected (and peer- 
reviewed) studies in favor of reports 
from Friedman and other voucher ad- 
vocacy organizations. And on the rare 
occasion when it does refer to reputable 
research, it mischaracterizes those find- 
ings in order to support its own agenda. 

For instance, the report claims that 
“there is a large body of high-quality 
empirical evidence showing that vouch- 
ers make public schools better, not 
worse” (p. 10). It is a stretch to say that 
this literature is “large” — most observ- 
ers would agree that much more atten- 
tion has been focused on the immediate 
question of whether choice improves 
academic outcomes for the students ex- 
ercising choice. And the assertion of 
“high-quality” is itself highly question- 
able; indeed, “research” is not defined. 
For example, the report contends as fol- 
lows: 

Numerous fiscal studies have 
examined whether vouchers and 
tax-credit scholarships (a si mi lar 
type of school choice program) 
“drain money” from public 
schools. This body of research 
has shown consistently that 
these programs save money both 
for state budgets and for local 
public school districts, even af- 
ter the fixed costs of public 
schools (costs that do not go 
away when students leave a 
school) are taken into account 

(pp. 10-11). 



But it is not clear which studies are be- 
ing cited. In a footnote, the report indi- 
cates that “Most of these studies are 
available in the research database 
hosted on the Friedman Foundation’s 
website (w w w . friedmanfoundation . org 
/friedman/research/ShowResearch.do) ” 
(p. 17, Note 1). 

However, this links to all of the re- 
search the Friedman Foundation posts 
on multiple topics. Presumably, the re- 
port is referring to the “fiscal impact” 
category, which lists about two dozen 
“studies,” many focused on single 
states. All of these were produced by 
the Friedman Foundation, and none 
were peer-reviewed. 

Regarding the question of “saving 
money” (on which the report tries to 
make generalizations), the answer ob- 
viously depends on individual program 
design — programs could be structured 
to either protect or penalize public 
schools if they lose students. Indeed, it 
is puzzling as to why the report would 
assert that public schools are not 
harmed, since the report’s central ar- 
gument depends on public schools feel- 
ing real penalties from the loss of, or 
threat of losing, students — as indicated 
with the report’s discussion of the DC 
voucher program. That is, the threat of 
lost students and funding is the key 
mechanism by which market competi- 
tion might drive a response from public 
schools. 

In any case, many of the studies appear 
to be unrelated or marginally related to 
the topic, or are not studies at all (for 
instance, one “study” is a statement to 
legislators). A handful of them focus on 
topics such as graduation rates or pub- 
lic-private school achievement, with 
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only tangential conjectures about the 
introduction of choice programs. Still 
others, rather than providing empirical 
analyses of actual data, simply make 
predictions about non-existent choice 
programs that the reports propose. (I 
have critiqued such fiscal impact “re- 
search” from the Friedman Foundation 
previously. ) According to the Fried- 
man Foundation’s own criteria, these 
are not “high-quality” studies. 

To further support the claim that 
voucher competition improves public 
schools, the report cites five studies on 
the voucher program in Milwaukee, 
nine on Florida, and two on other states. 
The report celebrates the prestigious 
institutional affiliations of the research- 
ers, although most of the studies were 
actually produced by pro-voucher think 
tanks, including the Friedman Founda- 
tion, Manhattan Institute, and the Hoo- 
ver Institute. Of the 16 studies (one of 
which is cited twice^^), five are unpub- 
lished working papers. Only two were 
conducted by respected researchers not 
associated with a pre-announced agenda 
regarding vouchers, and only one was 
published in a reputable peer-reviewed 
journal. Moreover, many of these 
studies conducted by voucher advocates 
have been questioned in the scholarly 
community. For instance. Professor 
Hoxby’s study of competition in Mil- 
waukee was criticized for using a 
school-level analysis (as this report 
does) rather than a student-level analy- 
sis, and was published in a journal of 
“opinion and research” produced by 
pro- voucher think tanks. 

The concern here is not the citation of 
work published outside rigorous peer- 
reviewed outlets. Even advocacy ven- 
ues can produce high-quality research. 



Rather, the problem is that such a small 
fraction of the research has ever even- 
tually made its way through such a rig- 
orous peer-review process. 

The Friedman report selectively focuses 
on studies — no matter what the qual- 
ity — that appear to support its agenda. 
In doing so, it leaves out much high- 
quality research, much of it peer- 
reviewed, that seriously questions the 
assertion that the threat of losing stu- 
dents has a positive impact on public 
schools. Much of this work has been 
done on charter schools, which are 
much more widespread than the 
voucher programs that the report high- 
lights. These studies explore dynamics 
comparable to those created by voucher 
policies, examining schools facing the 
threat (or real) loss of students. The 
overall results are mixed, but some of 
these studies identify essentially no, or 
even negative, competition effects on 
public schools. The report’s discus- 
sion of competition literature would be 
considerably stronger if it had included 
such research. 

Finally, the report misrepresents find- 
ings in order to support its assertions. 
For instance, it cites a recent study by 
Carnoy et al. to lend support to its 
claim that “These studies unanimously 
found that public schools improve when 
a voucher system has been imple- 
mented” (p.ll). This is a curious claim. 
Carnoy and his colleagues report on 
their two-part study, and neither part 
supports the Friedman conclusions. The 
first part was basically a replication of 
earlier studies, finding initial public 
school gains but also finding that those 
gains then fell off — suggesting that dif- 
ferent types of market conditions need 
to be considered, as do alternative ex- 
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planations for gains. In the second part, 
they adopted a more nuanced approach, 
and found “essentially no evidence that 
students in those traditional public 
schools in Milwaukee facing more 
competition achieve higher test score 
gains.” The Friedman report misrep- 
resents those major findings. 

V. Review oe the Report’s 
Methods 

The report uses a regression analysis, 
controlling for basic demographic fac- 
tors as well as for the presence of char- 
ters. Rather than tracking individual 
student gains over time, which is a 
preferable method for measuring school 
effects, the analysis uses school-level 
data. (This is unfortunate, because 
mobility between schools might mean 
the report is measuring substantially 
different populations within the same 
school from year to year — with these 
data we simply do not know.) Setting 
aside the use of school-level data, the 
regression approach is generally appro- 
priate. But the analysis ultimately fal- 
ters on some serious methodological 
missteps, fatally damaging the study. 

The report is based on two compari- 
sons: first, achievement growth for 

“voucher-eligible” schools compared to 
other Ohio public schools, and then, to 
check for the possibility of regression to 
the mean, it compares “voucher- 
eligible” schools in “very poor urban 
districts” to other public schools in such 
districts. This second comparison using 
a district- wide scope is problematic, 
since school quality and resources can 
vary substantially within districts. That 
is, the report’s approach implicitly as- 
sumes that the comparison district 
schools should also be very low scoring 



due to demographics — about the same 
as the voucher-eligible schools. But 
there is, by definition, something about 
these non-“voucher-eligible,” compari- 
son schools that are also in poor dis- 
tricts that distinguishes them from 
“voucher-eligible” schools in those dis- 
tricts. This something might be leader- 
ship, teaching staff, curriculum, or the 
demographic backgrounds of students. 
Forster desperately wants to assume 
that vouchers are making up the differ- 
ence, but it is likely that he is compar- 
ing the already more-effective schools 
to schools that had more room for im- 
provement. Indeed, those differences 
between the two sets of schools would 
have pre-dated the introduction of the 
voucher program. A more useful com- 
parison would be to examine student 
academic growth and instructional pro- 
grams of individual schools where stu- 
dents are eligible for vouchers relative 
to demographically similar schools that 
are more academically successful in 
order to also consider other factors that 
could be responsible for greater school 
effectiveness. 

Another problem is that the demo- 
graphic measures used in this report are 
extremely weak. The analysis controls 
for the percentage of white students at a 
school, as well as the percentage of stu- 
dents eligible for free or reduced-price 
lunch. This white versus other approach 
prevents the analysis from distinguish- 
ing among African-American, Asian, or 
Hispanic students, and yet these are im- 
portant considerations. (For example, 
Hispanic students might be more likely 
to apply for a voucher to attend a 
Catholic school, and their academic 
gains often differ from those of AM- 
can-American students; and pro- 
voucher research, even in the best light. 
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has usually found an effect only for Af- 
rican-American students. Further- 
more, the free or reduced-price lunch 
classification is often regarded — 
especially by voucher proponents such 
as Forster^^ — as an inappropriate vari- 
able in itself, which it can be when used 
as a sole measure of SES, as is the case 
in this report. 

The report does make the reasonable 
decision to control for the presence of 
charter schools, presumably under the 
assumption that the availability of this 
additional option by which students can 
leave a school could confound the re- 
port’s attempt to measure the effect of 
the possibility of exit through vouchers. 
Yet the report only looks at the number 
of charter schools as a percentage of all 
schools in various cities. A more useful 
approach would have looked at market 
share: the percentage of students or 
spaces in charter schools. This is impor- 
tant because charter schools tend to be 
smaller schools, so school counts used 
in isolation could over-estimate charter 
competition. Moreover, it is unclear 
why the report would account for char- 
ter competition but fail to consider the 
number of private schools, especially 
those accepting vouchers, and the num- 
ber of available spaces in those schools. 
There would be no threat of exit to a 
failing public school if there are no pri- 
vate schools nearby (or none that accept 
vouchers), or if the private and charter 
schools are full. Thus, between these 
factors, not to mention the availability 
of open-enrollment options (that is, the 
choice to enrollment to a public school 
other than one’s neighborhood school) 
in Ohio, it is nearly impossible to de- 
termine if EdChoice and the exit option 
it provides are the cause of any changes 
in achievement in public schools. 



Einally, the report is concerned about 
disproving the possibility that “regres- 
sion to the mean” is influencing any 
apparent gains in achievement — that a 
school’s low score in Year 1 would in- 
crease the next year as a result of a sta- 
tistical artifact, rather than as a result of 
true gains. Although this is a valid con- 
cern, the possibility is present only be- 
cause the analysis used differences in 
achievement between Year 1 and Year 
2 as the dependent variable. Instead, the 
analysis should have used the Year 1 
test as a predictor for Year 2 outcomes, 
as the author of this report has done 
elsewhere, but strangely not here.^^ 
This preferred approach would control 
for the fact that some schools are start- 
ing Year 1 very high or very low, and 
thus have more “room” to make gains 
(or more limits, in the case of a ceiling 
effect for initially high-scoring 
schools). This more sophisticated ap- 
proach would account for any tendency 
of regression toward the mean, much 
more so than the simplistic approach 
used in the Eriedman Eoundation re- 
port. 

VI. Review oe the Validity 
OE THE Findings and 
Conclusions 

Perhaps the greatest weakness of the 
report is its interpretation of the results. 
The report concludes that its analysis 
demonstrates “substantial beneficial 
effects on academic outcomes in public 
schools from EdChoice vouchers, and 
no harmful effects” (p. 13). But these 
benefits are hardly clear. Consider the 
results of the two comparisons in Ta- 
bles 1 and 2 of the report. Regressions 
were run on growth at five grade-to- 
grade stages (i.e., 3'^'^ to 4*) separately 
in reading and mathematics, for two 
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different analyses, giving us a total of 
20 coefficients for which there could be 
a voucher effect. Of those 20, 6 show a 
statistically significant positive effect 
(grades 4-5 and 6-7 in math, and grades 
6-7 in reading in both analyses), with 
gains in math (-1-5 scale points) outpac- 
ing those in reading (-1-2). Yet there was 
no statistically significant effect for the 
vast majority of grades. Based on these 
highly uneven and inconsistent results, 
the report argues for a cumulative effect 
over time that can close the achieve- 
ment gap by one standard deviation in 
four years. 

Actually, ten of the twenty outcomes 
are either zero or negative, and those, 
along with four other positive effects, 
are not statistically significant at the 
95% (or .05) level. Here it should be 
noted that the report’s author has in- 
sisted elsewhere (in arguing for evi- 
dence of academic outcomes) that sta- 
tistical significance at the 95% level of 
certainty is not a cut-point but an arbi- 
trary convention; he has argued instead 
for a continuum of confidence. If we 
accept this logic, the report has to some 
degree disproved its own claim that 
vouchers have “no harmful effects” on 
public schools (p. 13), since the results 
indicate a negative five -point effect in 
math from grades 5-6 in one analysis, 
with 89.2% certainty that it was not just 
statistical “noise” (p. 15). 

The report asserts that vouchers have a 
“substantial” effect despite the fact that 
they do not have any measurable effect 
in the majority of situations analyzed. 
Further, Table 3 of the report cherry- 
picks the three findings where the 
analysis was able to get positive find- 
ings that were also statistically signifi- 
cant. This is inappropriate. In view of 



the inconsistency of the results, the pat- 
terns strongly suggest that there are 
other unmeasured factors at play here 
and that the three positive results are a 
fluke. In fact, the report should explain 
to readers why there was no effect (and 
possibly even a sizable negative one) in 
the vast majority of cases. 

The report tries to build an even bigger 
case on even this paltry evidence, 
claiming: “If the effects accumulate 
over time, in three to four years the 
voucher-eligible schools will have im- 
proved by one standard deviation (equal 
to one-sixth of the distance between the 
top-scoring and bottom- scoring schools 
in Ohio)” (p. 5). Yet the report offers no 
support for this contention that any 
gains are cumulative. And even if this 
optimistic and unfounded assessment 
were correct, that means that a K-12 
voucher program would not close the 
achievement gap between the lowest 
and highest scoring schools in Ohio. 

VII. Usefulness of the Report 

FOR GumANCE OF POLICY 

AND Practice 

In addition to this and other reports, the 
Friedman Foundation produces a 
“Guide to Evaluating the Scientific 
Quality of Education Research. The 
guide lists nine considerations in as- 
sessing research — issues such as alter- 
native explanations, generalizability, 
appropriate comparisons, and the avail- 
ability of other research. This new re- 
port arguably falls short on at least half 
of the Eriedman Eoundation’s own cri- 
teria. 

While this analysis draws on quantita- 
tive data and asks interesting questions, 
the quality of the empirical analysis 
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falls far short of the “highest standards 
of scientific rigor” promised by the 

•iq 

Friedman Foundation. In view of the 
announced advocacy mission of the 
Friedman Foundation for Educational 



Choice regarding vouchers, and the no- 
table flaws on this report, it is better 
read as a statement of belief than as an 
empirical analysis. 
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